This work attempts to provide a theory/definition for how to define a match between two clusters of neurons (e.g. two layers), each from a neural network. The definition (Definition 1, in paper) allows one to study the similarity between a pair of neural clusters (of the same size). In this paper, this definition of match is employed in this paper to study two networks of the same architecture but trained with different random initializations. This work seems to be a follow-up work from Convergent Learning [1] and SVCCA [2]. The authors characterize the representation of a neural cluster C as the subspace spanned by the set of activation vectors of C.

Engineering principles when coupled with a clear set of goals/constraints/behaviors can help guide the development of effective cognitive architectures.